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ABSTRACT 

This study represents one of the first major 
summative evaluations of the Science Curriculum Improvement Study 
(SCIS) , a federally funded, six-year elementary science program. An 
instrument was designed for measuring the effects of SCIS on the 
stated goal of the curriculum, the development of scientific 
literacy. Subjects were rural sixth-grade children: 312 had studied 
SCIS; 219 were otherwise comparable but had not studied SCIS. Besults 
indicated that SCIS contributed to the development of scientific 
literacy. Specific tasks which differentiated SCTS from non-SCIS 
students included the identification and control of variables, 
understanding relative position, and explaining energy transfer.. 
Begardless of SCIS experience, the girls outperformed the boys on the 
Scientific Literacy Test developed for this study. (Author/BW) 
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Abstract 



This study represents on of the first major surmatlve evaluation of 
the Science Curriculum Improvement Study (SCIS), a federally funded, six- 
year elementary science program. An Instrument was designed for measur- 
Ing the effects of SCIS on the stated goal of the curriculum, the develop- 
ment of scientific literacy. Subjects were rural sixth-grade children: 
312 had studied SCIS; 219 were otherwise comparable but had not studied 
SCIS. Results Indicate that SCIS contributes to the development of scien- 
tific literacy. Specific tasks which differentiated SCIS from non-SCIS 
students Include the Identification and control of variables, understanding 
relative position, and explaining energy transfer. Regardless of SCIS ex- 
perience, the girls outperformed the boys on the Scientific Literacy Test 
developed for this study. 



Iht reduction 

The development of scientific literacy among all segrt»nttf |>op""* 
latlon has been a goal actively supported by many Juring th^ pas" <jccades, 
Hultldlsclpllned teams of curriculum developerSf f^r eJ«ainpU^ wt<^^ ^^t^anclal 
support from the federal governttent, Jointly conceived a>\d tj^rrl^** ^U^. ex- 
tensive researchi-development projects so that chtl<lren "Ight 9^'^ ^ their 
imderstandtng of science. To b,* scientifically literate ha$ com^ ,^an 
that one needs to have: (0 a functional understariding <if tfie ^^^^ Ideas 
In science; (2) knowledge of th,s history of sclcnorj (3) cUhfosl<^^ '''^gard- 
Ing materials and events; and (<») an understanding of th^ ln«e'^''^'^^*Onshlps 
between science and society* 

Serious evaluation studies of the federally fl«ianc«<J cUhf'^"^*"^ Projects 
aided at developing scientific literacy are essential, Curh^tlV ^ ^g^j* 

mated thirty percent of the eleiDentary school chtl<Jren 'h th^ Un^**** States 

/al 

are using one of three currlcuU developed with th« al<te of /cdef ^und*' 
Science Curriculum Improvement Study (SCIS), Scleh<:e— A ^^©(SeS^ '*^'^*^ach 
(SAPA), and Elementary Science Study (ESS (Llvernx>re, f rl vat^ Cor^'*'^l,.gtlon) . 
All three programs use direct experiences rather tfian tentbticj''* ^* 
primary media for teaching elementary science. 

Although the materials-centered appraoch wa« sjjgge^t^d by Pe^**^^*2l 
as long as 150 years ago, and more recently by Mo»)fess<>rI and It was 

fit 

not until the major curriculum reform effort of th^ Ig^O', tj,at i * teach- 
ing approach came to be used wl()ely l»« elementary ^clef)C« ecJucat**'"* These 
non-textbook science curricula heed to be assessed to detefitifne ^'^ ^helr 
potential for developing a Scientifically literate popuUcei (2) l^i^ds 
of science concepts that can be effectively taught to *l«n*ncanr 
children through hands-on experience, and (3) the «ffetft jdei^^ o^, (he 
davolopment of children's ability to think logically. 
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SC*^ Was one of the major elementary science pronrans to grow out of 
the e^rty ]g60's when understanding of the structure of a discipline was. 
<JeeP«'^ utmost Inportance (Bruner, I963). The scientists, educators, 
and P^^'^^ologlsts Involved In the development of SCIS constructed a se- 
qtjent'^^ program for grades kindergarten through six which reflects the 
basic P'*lnc|ples of science and yet can be taught by ordinary teachers to 
ordln^*^ students. The stated goal of SCIS Is the development of scien- 
tific ^Ue racy (Karplus, ISfi**)* An overview of the manner In which the 
SCiS ''"''••Iculum atteiTpts to realize this goal Is described elsewhere 
(5Cls'''^«cher's Handbook, 197*). 

E^^^Uatlon studies concerning the effects of SCIS on children's reason- 
ing h^v^ fallen Into two broad categories: (l) those yielding data use- 
ful to designers of SCIS In making curriculum decisions and (2) those 
q|vln0 ^formation concerning the cognitive effects of particular units or 
on« two year's study of the program. 

S«"<*le$ of the effect of one or two SCIS units sug^st that positive 
effect* Can be measured In the areas of conservation (Haan, 1968; Stafford, 
1569) » ^erlal ordering <Almy, 1970), compensating variables (Linn and Thier, 
\$75)' ^Classification (Linn and Peterson, 1973), relative position and 
iW>tlon (fiattagllnl* 1972), and utilizing the processes of science (Weber, 
1972)' HowevC, these results cannot necessarily be generalized to all 
j>^ul«*^ons. For example, the socio-economic levels of the children being 
studl^^» the teaching style used, and the particular concepts being eval- 
uated P'^oved to be significant variables when analyzing effects of SCIS 
(tlon "f^d Peterson, 1973; Linn and ThIer, 1975? Haan, I968; Almy, 1970). 
Studl** assessing the cumulative effects of the total curriculum have been 

■ 5 :i 

o 

ERIC 



-3- 



almost nonexistent because of the lack, until very recently, of school 
systems that have used the pronram for six consecutive years. 

The purpose of the present study was to Investigate some of the cumu*- 
latlve effects of SCIS on chl1dren*s development of scientific literacy. 
Specifically, scientific literacy In the area of functional understandings 
of major Ideas !n science was assessed. It was hypothesized that children 
who had been exposed to SCIS would evidence a greater degree of scientific 
literacy than those who had not been exposed to the curriculum. A sclen-* 
tlflc literacy test was developed and administered to sixth graders In 
rural Hlchlgar; who had used the SCIS program for six years and to an equi- 
valent control group. 

Development of the Scientific Literacy Test 

Due to a lack of existing measures, a battery of tasks was designed for 
this study. Seven SCIS staff, three of whom had been with the program 
since Its Inception, contributed to the Initial phases of the test develop* 
ment. The tasks finally selected for Inclusion In the Scientific Literacy 
Test were Justified on the grounds that they represented major scientific 
concepts central to modem science and were Included In the SCIS curricu- 
lum. The following criteria were used: 

(1) tasks should be appropriate for eleven and twelve year old 
children; 

(2) tasks should lend themselves to a pencll-paper, whole class 
testing situation; 
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(3) tasks should form a representative selection of content- 
process concepts tauqht In the SCIS pronram; 

m) tasks should involve denionstratlons , pictures, or other con- 
crete referents; 

(5) tasks should allow for open-ended responses, whenever pos- 
sible, to permit evaluation of the children's reasoning; 

(6) whenever possible, appropriate tasks already In existence 
should be revised and used. 

Pilot tests were carried out In six classrooms. As a result p the 
seventeen originally suggested tasks were reduced to nine which composed 
the final form of thcL^cIentlf Ic Literacy Test. Although the test Is not 
totally comprehensive In terms of the major science concepts measured , It 
does Include a subset of basic scientific concepts appropriate for eleven 
and twelve year old children. 

The nine tasks are listed In Table I. Tasks I-IV examine children's 
thinking regarding some of the basic processes of science. Tasks V-IX 
attempt to measure the children's understanding of some of the major con- 
tent oriented concepts. 

An example of task from the test Is Analyzing Experlmants. In this 
task children are required to critique experiments In relation to given 
experimental questions, and/or results. The Analyzing Experiment Task 
does not reseiTi>le any SCIS activity In wording or equipment. A brief de- 
scription of the procedure and one Item will be given. 

7 
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The following Instructions were given to the exporlrnonters admlnlstei 
tng this task: 

"Tell your pupils that you will show them an experiment using the 
objects on your tray. Hold up the objects one at a time and name 
then (shoe box with a small hole In one end; one large balloon; one 
small balloon; fifteen straws). Insert one of the balloon s neck 
through the hole In the box so that the body of the balloon Is In- 
side the box. Partially blow up the balloon; arrange a track of 
eight to ten straws on a table; place the box on the track, and In- 
vite the students to predict what will happen when you release the 
balloon. Finally, let go of the balloon so the air escapesv caus- 
ing the box to move. How distribute the student page and explain 
that the picture shows the top view of an experiment similar to the 
demonstration. Read the questions aloud. Invite pupils to get 
clarification from you Individually as you walk among them. 



One Item from the Analyzing Experiments Task student pages distributed 
to each of the children Is shown In Figure 1. 

On the average, each of the nine tasks consisted of four Items. Often 
the Items vary wlthln-.a task In terms of the cognitive demands made of the 
child- Six of the nine tasks require explanations to accompany discrete 
answers; one task Is open-ended and allows the children to list as many 
responses as they wish; another gives two distinct problem-solving situa- 
tions assessed with a total of nineteen multiple choice questions. 

All of the tasks chosen for Inclusion In the Scientific Literacy Test 
require the children to perform mental operations. In the Plagetlan sense. 
In order to give evidence of a satisfactory understanding of the concepts. 
This means that a memorized answer or guess Is not sufficient. When writ- 
ten explanations are required It Is possible to Infer something about the 
children's thinking processes relative to the specific concept being eval- 
uated. The tasks, a description of what each measures, the specific pro- 
cedures for administration, and the SCIS activity most closely reseirbling 
the task, are reported elsewhere (BoMyer, 1975). 8 
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The validity of the Scientific Literacy Test was estimated by match- 
ing the objectives of the curriculum to the test Instrument. Table 2 
shows the relationship between the stated objectives In the SCIS curricu- 
lum and the content of the Scientific Literacy Test. 

It win be noted that the Scientific Literacy Test measures more of 
the stated curriculum objectives from the fourth, fifth, and sixth grade 
units than from the lower three grades. Also, the test Is somewhat 
weighted In the direction of the objectives of physical science units. 

To determine the reliability of the Scientific Literacy Test, test- 
retest scores for a group of twenty-four children who had experienced vary- 
ing degrees of SCIS but who were not used In the study were computed. The 
correlation coefficients ranged between .85 and .97. 

Methods 

Subjects 

A total of 531 middle-class, sixth-grade students from 19 classrooms 
In a rural area of Hlchtgan participated In this study. The experimental 
group consisted of 312 children who formed the entire sixth-grade population 
from two elementary schools; l?6 of these students or sixty-two percent of 
the school population had completed the six year curriculum program In Its 
entirety. 

The 219 children who served as controls were selected from three schools 
that were In close proximity of the experimental s" schools. The control 
schools for this study were chosen because they were educationally equiva- 
lent In term of socio-economic status and Intelligence (Brucet 1969). 

9 
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About half the sixth grade students In each school s^^re tested. Principals 
In both schools claimed that classes Irt their schools were heterogeneous I y 
grouped and that the students tested were ^ representative sample of their 
sixth-grade student populations. The control children spent the same amount 
of time on science per week, approximately one and one half hours, as the 
SCIS children. The text books used were: Concepts In Science (Brandweln, 
1967) and Sfclence In Our World (Schneider, 1961) • Scores for the Michigan 
State Assessment Tests (1971) for the two groups Indicated that the ability 
level of the control group was slightly higher than for the experlmentals 
(experimental mean - *9.9; standard deviation « 8.9; control mean - 51 .B; 
standard deviation - 8,7). The average age of the children In the two 
groups was the same: twelve years, three months. 

T eachers 

All the teachers In the experimental schools taught SCIS, not Just a 
selected group of better teachers. No systematic difference existed In 
the quality or type of teacher hired by the experimental and control school 
systems. The children tested represented the first group to have used 
SCIS at each of the different grade levels. It should be pointed out, ho%/- 
ever, that the teachers* lack of experience may have been compensated for 
by beginners* enthusiasm* 

The two experimental schools used In this study served as SCIS trial 
centers. These teachers (1) provided feedback to the curriculum developers 
for use In revising the program, (2) attended an Initial workshop prior to 
their teaching the science program, and (3) received In«*servlce training 
throughout their first year of using SCIS. 

10 
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Task Administration 

The nine Individual tasks were organized Into five packets, each con- 
taining from one to three tasks. These were distributed In random order 
to prevent any systematic Interaction effects from occurring between tests. 
Four experimenters administered the group tests. The nineteen classes were 
each tested In two forty-five minute sessions on successive days. The . 
average amount of time allotted for each task was ten minutes. 

The experimenters were each given a standardized set of directions for 
the administration of the tasks. The recommended procedure for handling 
student questions concerning test Items was to answer each child Indivi- 
dually at his desk and refer him to the directions or questions on the 
student page. The experimenters were asked to record questions the students 
asked during the testing periods. The only task which consistently eli- 
cited questions was the Histogram - Task Ml. On the average, seven chil- 
dren from each class, regardless of SCIS exposure, requested Information 
regarding the test. 

The students* reactions to the testing situation ranged from overt 
enthusiasm to adamant refusal to work on the activities. Both of these 
attitudes were rare, however, and were not observed more frequently In 
either the experimental or control groups. The most usual student re- 
sponse was a positive, cooperative ^t I tude on the first day and a neutral 
resigned attitude on the second day. 

With two exceptions (Histogram Task III, Life Requirements Task VMl), 
the students either first observed a demonstration experiment using mate- 
rials relating to the pictures and text on the test pages or hear a descrip- 
tlon of an experiment accompanied by a presentation of associated data. 
The experimenter then read orally the student pages to the entire class. 

11 



The time allowed for each task appeared sufficient for the majority of the 
children to complete all of the Items. After the alloted time, the exper- 
imenter collected the papers or asked the children to turn to the next task 
In their packet and the sequence was repeated. 

Scoring 

Scoring criteria was established by the authors. Two persons separately 
scored three control and experimental classes which served as the raw data 
for the reliability coefficients. Neither knew the group Identity of the 
papers scored. The remaining sixteen classes were scored by only one person. 
Scorer reliability was determined to be at the 92* level of agreement. 

Results and Discussion 

Individual scores from the battery of nine tasks (Table 2) given to the 
312 children who studied SCIS (experimental group) and 219 children who did 
not study SCIS (control group) served as raw data for a multivariate sta- 
tistical analysis. Multivariate Analysis of Variance was used In this study 
because scientific literacy could not effectively be encompassed by a single 
dependent variable (McCall, 1970). The nine task scores were treated as 
dependent variables; sex and SCIS experience were the Independent variables. 

Overall, the children with SCIS experience performed better on the 
Scientific Literacy Test than those without SCIS experience (F ■ 13.7; 
df - 9 and 519; p < .0001). The results of the imiltlvarlate analysis are 
shown In Table 3. 

The tasks which accounted for most of the /difference In performance 
were Variables, Analyzing Experiments, Relative Position, Energy Transfer, 

12 
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and Solution R Evaporation. Each of the?e tasks was slgi^lflcant at least 
at the p < .01 level. The Histogram task was significant at the .05 level 

In favor of the non-SCIS performance. An analysis of thci SCIS task score 

I 

means, according to the number of years the children studied the curricu- 
lum, did not reveal any differences due to length of SCIS exposure. 
SCIS had the greatest effect In these six areas: 



(1) Identifying significant variables In an experiment (Task I). 

(2) Detecting Inadequacies of Improperly controlled experiments 
and suggesting Improvements In an experlmsntal design (Task II) 



(3) Explaining experimental results In terms of compensating var- 



iables (Task II). 



! 



(h) Mentally putting oneself In another reference frama to commun- 
icate posltons of objects (Task iV). 

5) Predicting amounts of transferred energy as Indicated by changes 
In tenperature; predicting and explaining temperatures of equal 
mixtures of warm and cold water (Task VII). 



(6) Using the solution concept to solve a simple proportionality 
problem (Task IX) • 



A more detailed description of the data analysis 
(Bowyerp 1975). 

13 



Is reported elsewhere 



Sex Differences 

The girls outperformed the boys on the Scientific Literacy Test (F - 
*i.07; df - 9 and 519; p < .001). The results for the differential per- 
formance In terns of sex are shewn In Table 

The tasks which accounted for most of the sex differences were Vari- 
ables, Analyzing Experiments, and the Histogram Task which were signifi- 
cant at the .01 level. Further analysis revealed that the SCIS curriculum 
did not affect one sex more than the other (F - 1.2; df - 9 and 519; 
p < .28). 

Research Indicates that girls are superior to boys In verbal skills at 
the ages of twelve and thirteen (Maccoby, 1966). The verbal requirements 
of the tasks In the Scientific Literacy Test were analyzed and the results 

arc shown In Table 1. 

As can be seen, success on the Variables Task I depended upon the form 
and volume of the written language. Analyzing Experiments Task II required 
the greatest number of written explanations and the most reading. The 
Histogram Task III was the second most demanding In terms of the reading 
requirements. Thus the three tasks which accounted for the sex differences 
were the most verbally demanding. 

Implications 

The overall results of the study Indicate that northern rural children 
e}qx>fed to the SCIS program assimilate tome fundamental Ideas of science 
which are reflective of the basic structure of the discipline. Implicit In 
this Is the fact that SCIS contributes to these children's scientific 
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literacy development. From an educational point of view* these results 
have Important Implications which need to be considered. 

FIrstp the data supports and extends the results of other studies which 
Indicate that It Is possible to affect chl Idren's thinking during the five 
to thirteen age period with carefully selected manipulative activities. As 
a group, sixth graders who had direct science experiences were more sue* 
cessful In pencil-paper, problem solving tests requiring logical and sclen* 
tif Ic thinking than those who used textbooks. 

A second point Is the amount of classroom Input necessary to affect 
children's scientific and logical thinking. The SCIS InterventloOp though 
extended over a six year time period, represents only about five percent 
of the total teaching time, the same amount of time devoted to the study 
of science In the textbook oriented control classes. To detect effects of 
SCIS on children's reasoning abllltlesp given the relative size of the 
curriculum Input, Is Impressive. 

A third educational Implication concerns the fact that not all science 
concepts are understood with equal success. For exampley although the 
SCIS children as a group did significantly better than the non*SCIS chil- 
dren on the Energy Transfer Task VI I , half gave no evidence of understand- 
Ing thennal energy transfer In spite of the fact that three SCIS activities 
In the fifth year directly relate to the Item In this task. 

In contrast, solution, another science concept that was evaluated In the 
Scientific Literacy Test (Task IX), proved to be very %ie11 understood by the 
SCIS children. Three-fourths could predict and explain what would be left 
after the liquid from a salt water solution evaporated. In addition^ two- 
thirds could reason about the relative amounts of salt remaining after the 

15 
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llquld from two solutions evaporates. In contrast, this application of the 
Mlutlon concept to solve a simple proportionality problem was attenpted 
by only ^7Z of the non-SCiS children. It can be concluded that direct ex- 
perience with science concepts does not Insure understanding. 

A final consideration concerns the small absolute differences between 
the SCIS and non-SCIS children on the Scientific Literacy Test tasks.. One 
maasure of the educational Importance of the performance differences Is the 
length of time that they can be detected In children's thinking. The data 
from two tasks In this study (Analyzing Experiments Task 11 and Relative 
Position Task V) Indicate that curriculum effects In these two areas of 
logical thinking can be detected for at lejist tvfo /years after they w©ra 
taught. 

Linn and Thier (1973) showed that rural fifth graders who studied the 
Energy Source unit In SCIS are superior In using compensating reasoning In 
variables problems than similar children not exposed to the curriculum. 
One of the Items In the Scientific Literacy Test (Task II) also measures 
this ability and It was found that the curriculum effect Is still evident 
at the end of the sixth grade. Although the task employed by Linn and 
ThIer was different from that used In this study. Identical scoring cate« 
gorles were used. 

In m second task*. Relative Position, another example of sustained ln« 
terventlon effects Is evidenced. This notion has been extensively explored 
from a developmental point of view by both Plaget and Inhalder (1967) and 
Flavell and Botkin (1968)« 

In an Intervention study, Bataggllnl (1972) found positive transfer 

effects of the SCIS fourth grade Relative Position unit lirmedlately after It 

« 
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had been taught. Our research shows that children, tv/o years after their 
SCiS study of the concept, are still better able to describe positions of 
objects from non**self reference frames than comparable children who have 
not had the unit. 

It should be noted that the results of this study are limited In terms 
of the population to whom they can be generalized. The group of SCIS chll* 
dren evaluated were primarily from a rural section of the northern United 
States. It has been shown In other studies that rural children tend to 
score lower In science than do their suburban counterparts (Linn and Thier, 
1975; N.A.E.P., 1972). 

Further research needs to be conducted to determine (I) If these find- 
ings can be extended to other populations of children and (2) whether the 
SCIS differences are still In evidence In high school and college students. 
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Figure 1. Student Sheet for Task 1? 
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